what

• Adds missing filter_prefix and filter_suffix options in event notifications config for queues and topics.
• set lambda list filter arguments to optional

Signed-off-by: Leonid Bogdanov <leonidbogdanov86@​gmail.com>

references

• Closes #​260
• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3\_bucket_notification#lambda_function

what

• add s3 request payment config

why

• make requester pay

references

• closes #​156
• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3\_bucket_request_payment_configuration
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html

what

• support eventbridge bucket notification

why

• Emit s3 events to eventbridge

references

• Closes #​254

what

• use new destination.bucket key in policy

why

• Support both deprecated destination_bucket and new destination.bucket
• Previous changes created destination.bucket and left destination_bucket for backwards compatibility, as stated in variables.tf, and forgot to include the new value in the IAM policy

references

• Closes #​215

what

• use new destination.bucket key in policy

why

• Support both deprecated destination_bucket and new destination.bucket
• Previous changes created destination.bucket and left destination_bucket for backwards compatibility, as stated in variables.tf, and forgot to include the new value in the IAM policy

references

• Closes #​215

what

• Fixes replication IAM role name that gets created as '-replication'

why

• Replication IAM role name gets created as '-replication'. This made the replication role unsusable.
• Used Terraform version: 1.8.0

references

• Resolves #​251

what

It seems there is a typo kind if error here

dynamic "lambda_function" {
    for_each = var.event_notification_details.lambda_list
    content {
      lambda_function_arn = lambda_function.value.arn
      events              = lambda.value.events
      filter_prefix       = lambda_function.value.filter_prefix
      filter_suffix       = lambda_function.value.filter_suffix
    }
  }

I think it should be

dynamic "lambda_function" {
    for_each = var.event_notification_details.lambda_list
    content {
      lambda_function_arn = lambda_function.value.arn
      events              = lambda_function.value.events
      filter_prefix       = lambda_function.value.filter_prefix
      filter_suffix       = lambda_function.value.filter_suffix
    }
  }

why

The S3 notification can not be created unless this is fixed

references

This should fix https://github.com/cloudposse/terraform-aws-s3-bucket/issues/252

what

It seems there is a typo kind if error here

dynamic "lambda_function" {
    for_each = var.event_notification_details.lambda_list
    content {
      lambda_function_arn = lambda_function.value.arn
      events              = lambda.value.events
      filter_prefix       = lambda_function.value.filter_prefix
      filter_suffix       = lambda_function.value.filter_suffix
    }
  }

I think it should be

dynamic "lambda_function" {
    for_each = var.event_notification_details.lambda_list
    content {
      lambda_function_arn = lambda_function.value.arn
      events              = lambda_function.value.events
      filter_prefix       = lambda_function.value.filter_prefix
      filter_suffix       = lambda_function.value.filter_suffix
    }
  }

why

The S3 notification can not be created unless this is fixed

references

This should fix https://github.com/cloudposse/terraform-aws-s3-bucket/issues/252

what

• Update terratest >= 0.46.0

why

• Support OpenTofu for testing

References

• https://github.com/gruntwork-io/terratest/releases/tag/v0.46.0
• DEV-374 Add opentofu to all our Terragrunt Testing GHA matrix

what

• Update .github/settings.yml
• Update .github/chatops.yml files

why

• Re-apply .github/settings.yml from org level to get terratest environment
• Migrate to new test account

References

• DEV-388 Automate clean up of test account in new organization
• DEV-387 Update terratest to work on a shared workflow instead of a dispatch action
• DEV-386 Update terratest to use new testing account with GitHub OIDC

what

• Update .github/settings.yml
• Drop .github/auto-release.yml files

why

• Re-apply .github/settings.yml from org level
• Use organization level auto-release settings

references

• DEV-1242 Add protected tags with Repository Rulesets on GitHub

what

• Update .github/settings.yml
• Drop .github/auto-release.yml files

why

• Re-apply .github/settings.yml from org level
• Use organization level auto-release settings

references

• DEV-1242 Add protected tags with Repository Rulesets on GitHub

references

closes #​243

Issue - GH-239

what

This feature will make s3 event notifications which will have 3 options to trigger lambda or queue or topic so we can define a resource from this documentation.
aws_s3_bucket_notification

we also added s3 directory bucket which is a new feature in aws so addition of optional resource can be done if someone needs to use that with the help of terraform
aws_s3_directory_bucket

why

• Enhanced Event-Driven Architecture: The introduction of S3 event notifications allows the S3 bucket to trigger Lambda functions, SQS queues, or SNS topics. This facilitates seamless integration with other AWS services and enables real-time processing of data, which is crucial for building event-driven architectures.

• New AWS Feature Adoption: The addition of the aws_s3_directory_bucket resource reflects the latest AWS capabilities, ensuring that our infrastructure is up-to-date with current AWS offerings. This optional resource allows users to leverage new AWS features as they become available, promoting flexibility and future-proofing our Terraform configurations.

• Improved Flexibility: By providing options to trigger different AWS services (Lambda, SQS, SNS), the solution becomes more versatile, catering to a wide range of use cases and workflows. This flexibility can lead to more efficient and effective data processing pipelines.

• Reduced Operational Overhead: Automating responses to S3 events using Lambda functions, queues, or topics can significantly reduce manual intervention and operational overhead. This leads to improved efficiency and allows teams to focus on higher-value tasks.

references

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3\_bucket_notification
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3\_directory_bucket
https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html

what

Add tags to resources where they missed

what

This PR addresses #​226 about not being able to specify expected bucket owner in various S3 resources.

why

From AWS docs:

Because Amazon S3 identifies buckets based on their names, an application that uses an incorrect bucket name in a request could inadvertently perform operations against a different bucket than expected. To help avoid unintentional bucket interactions in situations like this, you can use bucket owner condition. Bucket owner condition enables you to verify that the target bucket is owned by the expected AWS account, providing an additional layer of assurance that your S3 operations are having the effects you intend.

references

#​226

what

This variables adds a policy to the bucket to deny connections that do not use TLS 1.2 or higher.

why

This is required by our security team.

references

https://repost.aws/knowledge-center/s3-enforce-modern-tls

Bumps github.com/hashicorp/go-getter from 1.7.1 to 1.7.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps github.com/hashicorp/go-getter from 1.7.1 to 1.7.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

what

• Update workflow (.github/workflows/release.yaml) to have permission to comment on PR

why

• So we can support commenting on PRs with a link to the release

what

• Update workflows (.github/workflows) to use shared workflows from .github repo

why

• Reduce nested levels of reusable workflows

what

• Update workflows (.github/workflows) to add issue: write permission needed by ReviewDog tflint action

why

• The ReviewDog action will comment with line-level suggestions based on linting failures

what

• Update workflows (.github/workflows/settings.yaml)

why

• Support new readme generation workflow.
• Generate banners

Bumps golang.org/x/net from 0.8.0 to 0.23.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

what

• Install latest GitHub Action Workflows

why

• Use shared workflows from cldouposse/.github repository
• Simplify management of workflows from centralized hub of configuration

what

• Install a repository config (.github/settings.yaml)

why

• Programmatically manage GitHub repo settings

what

This is an auto-generated PR that updates the README.md and docs

why

To have most recent changes of README.md and doc from origin templates

what

• Reran make readme to rebuild README.md from README.yaml
• Migrate to square badges
• Add scaffolding for repo settings and Mergify

why

• Upstream template changed in the .github repo
• Work better with repository rulesets
• Modernize look & feel

what

• Allows users to specify a list of source IP addresses from which access to the S3 bucket is allowed.
• Adds dynamic statement that uses the NotIpAddress condition to deny access from any IP address not listed in the source_ip_allow_list variable.

why

Use cases:

• Restricting access to specific physical locations, such as an office or home network

references

what

Replaced the count with a for_each inside aws_s3_bucket_logging.default

there's no point in the try since the type is clearly defined as list

why

When the bucket_name within logging attribute is dynamically defined, like in the case of referencing a bucket created by terraform for logging

  logging = [
    {
      bucket_name = module.logging_bucket.bucket_id
      prefix      = "data/"
    }
  ]

we get this error

For each can work better in this case and will solve the previous error

references

what

This is an auto-generated PR that updates the README.md and docs

why

To have most recent changes of README.md and doc from origin templates

what

This is an auto-generated PR that updates the README.md and docs

why

To have most recent changes of README.md and doc from origin templates

what

This is an auto-generated PR that updates the README.md and docs

why

To have most recent changes of README.md and doc from origin templates

what

• When trying to create an S3 bucket, the following error is encountered:

Error: Invalid dynamic for_each value

  on .terraform/main.tf line 225, in resource "aws_s3_bucket_replication_configuration" "default":
 225:           for_each = try(compact(concat(
 226:             [try(rule.value.destination.encryption_configuration.replica_kms_key_id, "")],
 227:             [try(rule.value.destination.replica_kms_key_id, "")]
 228:           ))[0], [])
    ├────────────────
    │ rule.value.destination.encryption_configuration is null
    │ rule.value.destination.replica_kms_key_id is "arn:aws:kms:my-region:my-account-id:my-key-alias"

Cannot use a string value in for_each. An iterable collection is required.

• This is caused in my case by having s3_replication_rules.destination.encryption_configuration.replica_kms_key_id set.

why

• There is a bug when trying to create an S3 bucket, which causes an error that stops the bucket being created

  • Basically, there are two attributes that do the same thing (for backwards compatability)
    • s3_replication_rules.destination.encryption_configuration.replica_kms_key_id (newer)
    • s3_replication_rules.destination.replica_kms_key_id (older)
  • There is logic to:
    • A) use the newer of these two attributes
    • B) fall back to the older of the attributes if it is set and the newer is not
    • C) fall back to an empty array if nothing is set
  • There is a bug in steps A/B, where by selecting one or the other, we end up with the string value, and not an iterable
  • The simplest solution, which I have tested successfully on existing buckets, is to wrap the output of that logic in a list

• This error is easily replicable by trying compact(concat([try("string", "")], [try("string", "")]))[0] in the Terraform console, which is a simplified version of the existing logic used above

• The table below demonstrates the possible values of the existing code - you can see the outputs for value 2, value 3, and value 4 are not lists:

Breaking Changes

Terraform version 1.3.0 or later is now required.

policy input removed

The deprecated policy input has been removed. Use source_policy_documents instead.

Convert from

policy = data.aws_iam_policy_document.log_delivery.json

to

source_policy_documents = [data.aws_iam_policy_document.log_delivery.json]

Do not use list modifiers like sort, compact, or distinct on the list, or it will trigger an Error: Invalid count argument. The length of the list must be known at plan time.

Logging configuration converted to list

To fix #​182, the logging input has been converted to a list. If you have a logging configuration, simply surround it with brackets.

Replication rules brought into alignment with Terraform resource

Previously, the s3_replication_rules input had some deviations from the aws_s3_bucket_replication_configuration Terraform resource. Via the use of optional attributes, the input now closely matches the resource while providing backward compatibility, with a few exceptions.

• Replication source_selection_criteria.sse_kms_encrypted_objects was documented as an object with one member, enabled, of type bool. However, it only worked when set to the string "Enabled". It has been replaced with the resource's choice of status of type String.
• Previously, Replication Time Control could not be set directly. It was implicitly enabled by enabling Replication Metrics. We preserve that behavior even though we now add a configuration block for replication_time. To enable Metrics without Replication Time Control, you must set replication_time.status = "Disabled".

These are not changes, just continued deviations from the resources:

• existing_object_replication cannot be set.
• token to allow replication to be enabled on an Object Lock-enabled bucket cannot be set.

what

• Remove local local.source_policy_documents and deprecated variable policy (because of that, pump the module to a major version)
• Convert lifecycle_configuration_rules and s3_replication_rules from loosely typed objects to fully typed objects with optional attributes.
• Use local bucket_id variable
• Remove comments suppressing Bridgecrew rules
• Update tests to Golang 1.20

why

• The number of policy documents needs to be known at plan time. Default value of policy was empty, meaning it had to be removed based on content, which would not be known at plan time if the policy input was being generated.
• Closes #​167, supersedes and closes #​163, and generally makes these inputs easier to deal with, since they now have type checking and partial defaults, meaning the inputs can be much smaller.
• Incorporates and closes #​197. Thank you @​nikpivkin
• Suppressing Bridgecrew rules Cloud Posse does not like should be done via external configuration so that users of this module can have the option of having those rules enforced.
• Security and bug fixes

explanation

Any list manipulation functions should not be used in count since it can lead to the error:

│ Error: Invalid count argument
│
│   on ./modules/s3_bucket/main.tf line 462, in resource "aws_s3_bucket_policy" "default":
│  462:   count      = local.enabled && (var.allow_ssl_requests_only || var.allow_encrypted_uploads_only || length(var.s3_replication_source_roles) > 0 || length(var.privileged_principal_arns) > 0 || length(local.source_policy_documents) > 0) ? 1 : 0
│
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to
│ first apply only the resources that the count depends on.

Using the local like this

source_policy_documents = var.policy != "" && var.policy != null ? concat([var.policy], var.source_policy_documents) : var.source_policy_documents

would not work either if var.policy depends on apply-time resources from other TF modules.

General rules:

• When using for_each, the map keys have to be known at plan time (the map values are not required to be know at plan time)

• When using count, the length of the list must be know at plan time, the items inside the list are not. That does not mean that the list must be static with the length known in advance, the list can be dynamic and come from a remote state or data sources which Terraform evaluates first during plan, it just can’t come from other resources (which are only known after apply)

• When using count, no list manipulating functions can be used in count - it will lead to the The "count" value depends on resource attributes that cannot be determined until apply error in some cases

what

• Changed var.source_policy_documents to local.source_policy_documents so var.policy usage was still supported

why

• The ternary check uses var,source_policy_documents so var.policy being combined with var.source_policy_documents into local.source_policy_documents does not provide true for the ternary to execute

references

• https://github.com/cloudposse/terraform-aws-config-storage/pull/39